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ABSTRACT 

^ This review of the research on the evaluation of 

provides a review and synthesis of five major 
SJf ^ i^'SI prepared for and/or published for the United States 
Si^n! considers a number of problems and promises 

facing evaluators of career education in the eighties. Reviewed are 
reports prepared by TucKnan and Carducci in 197U, Development 
ll?B Sol!P: JS^^^^^Ju^" Ehaerman in 1 977, and Bonnet in 

lllrilJJrs t A 1^^^ ^"^^^ program within the Office of 

Education has devoted so much attention to evaluating its 
effectiveness on such a sustained basis. The overall ccnclusion 

i p «f^ro!l^^^*''''^^^^^^ evidence is more positive than negative. 
I.e., at present there is more reason to believe that career 
ro^r!ric''^i^ effective than that it isn't, your issues discussed as 
JrLt.T*M^.r*iyrf°'' eighties a- •. "the career education 

treatment (identifying what has been do in the name of "career 
education"): the need for immediate evaluation criteria for learner 

?o^«?InH^ Ji i^'L*''^ 2^^?^ control groups; and conducting 

longitudinal studies of the teaching- learning process. (MEK) 
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from the original document. 
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Introduction 



Concerns relat.ve to evaluating the of fe.t.veness ot career education have 
been prominent s.nce .nc-ept.on of the n,ovement ,n 1^70. Early attempts to 
summarise and synthesize the literature on this subject are illustrated by reports 
prepared by Tuc.man and Cardu. n . ,or the National Institute of Edncat.on in ,97. 
by neveiopment Associates for the Division of Vocational and Technical Education. 
U.S. Office of Education^ m 197.. s.nce OE's Office of Career Education was created 
by Congress in 197^,. several additional formal atte.npts to summarize and report 
0.1 studies concerned with evaluation of career education have been published. These 
.ncludinK one by Ender,ein\n 197.. one by Bhaerman^n 1977. and one by Bonnet"* 



and 



in 1978. 



'^'''''Z'^onZ: \ ^^^^^n^S^-^^^L^du^ A Review and Model 
Report No. 16. National Institute of Eten^irTmHEW? : Washington. D C 
U.S. Government Printing Office, 197.. ""Kion. u.i_.. 

^(^eveloprnent Associates. Inc. Evaluation^3LVSHia_tion al Exemplary P roject.- Part 

of Vocational i^^S^^^l^lFimU^l^u^.^U^^^^ Education. March" 197 5. 

^nderle.n Thomas AJjevjewj^LCdr^^ Evaluation Studies . Washington 
n.t .: U.S. <.uive^S;;T;i^?;rpFinting Office. 1976. wasnington, 

Wrman, Robert n. Caj^eer^Eclucanon Basic Academic A chiev^m.n,- ^ 

Winett, Deborah Ajvnt hesis of Resul ts and ProKram^^a tic Recommendations 
-HieroranPNun^^ 

of Education. Washington. D.C.: li.S. Government Printing O fi ce 1 9 8 
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In addition, Ot'H Otfire of Cdrecr Educdtion has had prepared, under contract, 

Iwo monographs • one by Mitchell^ and the second by Bonnet"^ - designed to provide 

modeis for usv by those involved in evaJuati.ig K-i2 Career education t?f tores. Further, 

OE's Off iceof Career Education has pubhshed two special monographs, one concerned 

with summarizing recommendations of invited experts on evaluation of career educa- 

Uot\ and the second summarising presentations on evaluation of career education 

9 

made at the 1976 Commissioner's National Conference On Career Education . 

It seems safe to say that never has such a small program, within the United 
Stales Office of Education, devoted so rtiuch attention to evaluating its effective- 
ness on Such a sustained basis as has been true for career education. Further, it 
also seems safe to say that, taken as a whole, the available evidence is more positive 
than negative - i.e., at present, it seems safer to say that career education can be effective 
than to sav it cannot bo efte('tive. This fact will be illustrated by reports to be pre- 
sented in this monograph. 

The interest of the ^\)ngros.s in the evaluation of career education continues 
and IS easily illustrated through study of P.L. 95-207, CAREER EDUCATION INCEN- 
TIVE ACT passed by C^ongress on December I 3, 1977. Sec. 5(a)(2)(C) of that act 

^Mitchell, Anita M. Ways to Evaluate Different Types Of Career Education Activities; 
A Handbook of Evaluation Models. Palo Alto; Americai^ Institute for Research, 
r978. 

^Bennett, Deborah C. Ev aluation Design and Reportinj; In Career Education. Wash- 
ington, D.C\; U.S. "Coverninent Printing Office, 1978. 

^Ioyt,K.^S. Perspectives On The Problem Of Evaluation In Career Education. Wash- 
ington, D.C: U.S. Government Printing Office, 1976. 

'\')avta, Lois-ellm et ai Career Education: What Pro of Do We Have That It W orks? 
Washington, D.C: l-.S. Covernment Printing Of fice, 1977. 
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call., for reserving an amount not to exceed one-haH of one percem of funds e?- 
propruted by Congress for purposes of carrying out a nat.oMal evaluat.on of the 
effectiveness of careex education programs receiving funds under the Act. 
Sec. 7(5) of the Act r«,u.res each State Education Agency receiving funds to pro- 
v.de proposed criteria to USOE for evaluating :he extent to which the State will 
achieve Its objectives under the State Plan. Sec. 8(a)(2KD) allows the State Educa- 
t.on Agency to use iCs State leadership funds under the Ac, for purposes of conduct- 
ing statew.de needs assessment and evaluation studies. Sec. 8(a)(3){M) allows local 
school districts receiving funds under the Act to use such funds, in part, for ccnduct- 
.ng needs assess.nents and evaluations. It is clear that, as we move into the decade 
of the 1^80s. the Congress desires that the emphasis on evaluatio,, of career educa- 
tion apparent during the decade of the 1970s be continued. 

One of the ma.or problems facing career education at the present time is that 
most Of the literature directly related to evaluation of career education has not 
yet found ,ts way into referred journals and/or books. Instead, much of ,t is in the 
form of project reports submitted to the U. S. Office of Education and documents 
published as monographs by the U.S. Government Printing Office. Such reports 
tend to become part of the "fugitive literature" not readily accessible to those in- 
terested .n the problem. Thus, the need for this monograph is seen, m part, as stemming 
from a desire to summan.e some of the earlier published documents m one place 
m order to try to .vo.d the "fugitive literature" problem to the greatest possible 



extent. 



A second Heed leading to production of this monograph us the need to reflect 
on experiences in evaluating career education durmg the decade of the 1970s in 
'10P..S that efforts conducted dunng the coming decade may profit from such ex- 
periences. 7hu<. the contents of th.s monograph are divided into two major sections. 
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In the first part, dn attempt will be made to provide brief revie\*'S of some of the 
major publications referred to t?arlier that have been prepared for and/or published 
by the United States Office of Education* The second major part of this monograph 
considers a number of problems and promises facing those who attempt to evaluate 
career education in the decade of the 1980$. 

Summary of Previously Reported Ev alu ation of t he Effectiveness 
Of Career Education Efforts At The K*12 Level 

As of the time this monograph is being written, limited copies of each of the 
documents to be reviewed in this section are still available. For those having access 
tp,$uch documents, it will obviously be desiraMe to study their entire 
contents carefully. Here, only very brief summary statements will be presented 
of what appear to be the critical content of each document along with brief re- 
actions to that content. Taken t*>gether, they represent most of the data currently 
available within OE's Office of Career Education pertinent to evaluation of 
career education efforts. 

Purposely omitted from this review are articles published in regularly published 
professional journals. It is assumed that interested readers can, if they choose, 
locate such articles for themselves. Also omitted from this summary are separate 
reports of doctoral dissertations concerned with the topic of evaluation of career 
education. These, too, are obviously available to interested persons who wish to 
locate them through such standard works as Dissertation Abstracts . 

Bhaerman, Robert D. Career Education and Basic Academic Achievement; A 
Descriptive Analysis of The Research . Washington, 
D.C.: U.S. Government Printing Office, 1977. 

The most comprehensive summary of efforts to evaluate career education 

m terms of its effectiveness m improving basic academic achievement was produced 

by Dr. Robert D. Bhaerman durmg the 1976-77 period during which Dr. Bhaerman 

ER?C ,. 
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wwked in OE'sOlf.ce of Career Education. This activity included an attempt to 
discover .nd analyze ail such attempts that had taken place during the 1970-75 period. 
A totai of 33 studies are identified and reviewed in the Bhaerman monograph. Of 
these, two were doctoral dissertations, seven (including one involving only one 
teacher) were from local K-12 school districts, eleven from county K-12 school 
systems, two from Statewide studies, two from the "Partners in Career Education- 
series of studies conducted in Texas, (our from OCE funded career education demon- 
stration projects, and the remainder from evaluation studies related to Experienced 
Based Career Education (EBCE) conducted under auspicies of various regional R&D 
laboratories. The search for reports of evaluation studies for inclusion in tnis mono- 
graph included an ERIC search as well as access to other reports that had been sent 
to OE's Office of Career Education during the 197^-76 period. 

Nature of the Studies: G enera l Su mmary Statement... Th> majority of these 
38 studies focused on mathematics (28 studies) and/or reading (23 studies). However, 
data were aiso reported on language arts (1* studies), studies skills studies), social 
studies (3) and science (2). Apparently, the term "basic academic achievement- 
has been used to describe each of these subject matter areas. It is obvious that, 
most commonly, it refers to reading and mathematics. Locations in which these 
studies were conducted were found throughout the Nation with the greatest number 
coming from the Northeastern States (17 studies) and the fewest from the far- Western 
States (k studies). The use of pre-tests and post-tests were seen in 25 of the 38 
studies With the use of control as well as experimental groups found in 13 studies. 
Measurement instruments were, for the most part, nationally used norm referenced 
tests of basic academic achievement. Sample sizes varied from as few as 5 pupils 
to.as many as several hundred. In terms of grade levels included, all grades, K-12 
were found with most of these studies including pupUs in Grades 3 - Grade 8. 
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Linrntdtions of the Studies; Surnmary Statements. Bhaerman points out the 
most commonly cited iimitafon^ given by authors of the reports he reviewed as 
indudingi (a) problems relating to establishing control and experimental groups; 
(b) problems relating to procedures for implementing the career education program 
(e.g., ''students did not receive a concentrated exposure"); (c) problems related to 
testing Procedures; and (d) identified problems related to the short time period covered 
by the project. In addition, in a list of ''general observations" found in the monograph, 
Bhaerman lists the following which could also be taken to represent some general 
limitations associated with the types cf studies he analyzed: (a) there was no apparent 
common operational meaning for the term "career education"; (b) a detailed description 
of the career education '^treatment" was often not clearly stated; (c) the scope of 
the studies varied widely; and (d) the quality of the reports also varied. 

The Career Education "Treatment". While some of the studies Bhaerman reviewed 
apparently did not specify the career education "treatment" with any degree of 
clarity or exactness, those that did so led him to make tlie following generalizations 
regarding the nature of that "treatment": (a) it is a longitudinal effort beginning 
in the early elementary school and continuing throughout the formal education 
system; (b) it emphasizes the process of career development; (c; over half of the 
studies indicated the development of specific curriculum guides, units, learning 
modu'es or lesson plans (or use m "infusing" career education concepts into regular 
classrooms; and (d) staff development for teachers was a common element in many 
of these studies as were use of community resources, field trips, site visits, and 
"hands on" experiences. The majority of these studies covered one academic year in 
length, but the range was large with iome projects involving 2-3 years and others 
periods as short as ^>-12 weeks. Bhaerman also reported that there seemed lO be 
a tendency (or those career education '^treatments" described most fully as comprehensive 
efforts to be asso<^iated with more statistically significant findings whereas those 
studios reporting mirumal and/or negligible effects of a career education "treatmefit" 

ERIC * 
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also tended to report the treatment itself as "minimal." 

In studying the detailed tables Bhaerman included to present a descriptive 
aniilysis of each study, it is most difficult to tell the extent to which the kind of 
career education "treatment'* outlined in the OE Monograph A Primer for Career 
Education was followed, but my general impression was that large parts of that 
"treatment" were missing in most of the studies included in this monograph. On 
the other hand, where "treatments " were described in some detail, the general 
impression one receives is that there iS a legitimate basis for calling what was done 
by the term "career education." The prime exceptions, of course, are those studies 
concentratmg on the Experienced Based Career Education Model. When this classic 
model is used, the proper term is "EBCE" but it is not "career education." 

Results; Of the 38 studies, 19 reported data indicating that reading and/oi 
mathematics was impacted at the .05, .01, or .001 level of significance favoring 
pupils who had been exposed to career education or, in the case of the EBCE studies, 
that the treatment students maintained their growth in the basic academic skills 
areas. Sixteen studies were moderately supportive in that they reported data which 
either indi$;ated reading and/or math grade equivalent score improvement, achieve- 
ment gains for both experimental and control groups, or varied impact. Three studies 
reported data which mdicated either negligible or minimal impact. 

Of the 19 studies Bhaerman labels as "generally supportive," the apparent 
meaning of that term is that, on at least one of the areas of basic academic achieve- 
ment included, statistically significant pre/post and/or control/experimental gains 
were registered for pupils exposed to the career education "treatment." Note that 
Bhaerman referred to such studies as "gt?nerally supportive" in spite of the fact 
that, within some of these studies, some of the comparisons failed to produce a 
positive effect for the career education ''treatment/' Of the 16 studies Bhaerman 
labels as "moderately supportive," that term appears to indicate that, in general. 
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whiJe the c-dre«r education treatment appeared to show some positive gams, such 
gams were typ.cally not greater than those observed among control group pupUs 
- I.e.. that career education didn't lower basic academic achievement over what 
would have occured had no such treatment been present. The three studies Bhaerman 
labels as "unsupportive" were ones where no impact was seeti. 

Conclusions: In his "conclusions" section of this monograph. Bhaerman makes 
several generalized statements. The foUow.ng quotations appear to represent the 
conclusions he reached; 

^tP'"'*.'^'^'*/" '^'I'^'^^e that the experimental groups did not con- 
sistently outperform their counterparts. Yet. the overall data is reassurme 

l.er 'X°' V ^'^'""^8 ''^^ picture, one can ^' 

observe the situation as quite favorable. There are some who have felt 
that career education would hurt academic achievement. That is cer ainJv 
not an interpretation this analyst would draw from the data " ^ 

"The overall observation I would make is cha academic achievement 
generally was either impacted positively or. when it was not. it did 
no interfere w.th that goal....I believe the general directions are 
positive and the overall tendencies are clear." 

My own conclusions, based on study of the "Data Charts" which appear in the 
Bhaerman monograph, lead me to concur f.-Uy with his statement that the evidence 
he summarized certainly makes clear the fact that the kinds of career education 
"treatments" represented in these studies give no indications that career education 
acts to depress academic achievement - i.e.. as Bhaerman says, "it doesn't hyn 
academic achievement." However, when I look closely at the "Data Charts" for 
Chose studies Bhaerman labels as "generally supportive." I find several where. ,n 
spite of the fact the career education "treatment" produced statistically significant 
differences ,n acadomir achievement of pupils in some areas, it failed to do so in 
others. There ,s no way of telling, with exactness, how many times the results were 
statistically significant and how n,any times they showed no statistically significant differences 



Hi 
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between pup.ls who were exposed to the carw-r HM.iratmn ••tr.-fltment" and ouDils who were 
not. Because of this, we do not know the extent to which those statistically signifi- 
cant differences reported are due to chance and how much they were really due 
to Hh? career education tredtm,.nt. Thus, these studies, while certainly promising, 
do not appear in any way to be conclusive. / 

In short. It seems to me these data tell us with a. reasonable degree of certainty 
that a career education treatment has not lowered basic academic achievement , 
but they do not tell us, with an equal degree of assurance, that the career education 
treatment will improve basic academic achieven^ent. The best they appear to be 
able to tell us is that the career education treatment can - and sometimes does - 
serve as a vehicle for improving basic academic achievement. 

If the data found m the Bhaerman mo/^ograph were to be interpreted in the 
most negative way, one could say that th^y leave open the question of the probable 
impact of a career education "treatmerri" on academic achievement. That is. findings 
showing no statistically significant differences between experimental and control 
pupils obviously may result from an/ one of three basic conditions including: (a) there 
really are no differences; (b) the t/me period and/or experimental conditions may 
have been inadequate to demonstrate that the experimental pupils actually do out 
perform the control pupils, or (c) the time periodand/or experimental conditions 
may have been inadequate to demonstrate that the control pupils actually out perform 
the experimental pupils. It is. of course, the third possibility that must concern us. 
It is very difficult to demonstrate, in a clear and convincing fashion, that an^. kind 
of "new" approach to the teaching/learning process actually improves academic achieve- 
ment. The typical finding is one of "no statistically significant differences." Typically, 
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a time period extending over several years is required in order to demonstrate clear 
and conclusive results. These studies, by and large, did not cover a tune period 
of over one year. Thus, we leave those who are "disbelievers*' with as much basis 
lor claiming that, in the long run, career education will be shown to have a negative 
effect on academic achievement as do the "believers" for claiming that, in the long 
ruOf It will have a positive effect. 

More than anything else, these data demonstrate to me the absolute necessity 
for finding and/or conducting a series of solid evaluation studies using randomly 
assigned experimental and control pupils under conditions of pre/post testing where 
assurances can be given that the kind of "career education treatment" recommended 
in A_j^rirner For Career Education is actually applied for a period of years with annual 
assessments called for. To date, no such comprehensive studies have yet been conducted 
to the best of my knowledge. They are certainly needed. 

!,- 

Enderlein, Thomas A Review of Career Education Evaluation Studies . Washington, 
O.C.: U. S. Government Printing Office, 1976. 

*hile serving as an intern in OE*s Office of Career Education during 1976, Dr. 
Thomas Enderletn was asked to compile a summary of available literature related to 
evaluation of K-12 career education efforts that had taken place prior to initiation of 
demonstration projects funded under provisions of P.L. 93-380, Section ^06 (which 
began during the year Dr. Enderlein made his search). His review is divided into three 
major sections; (a) career awareness; (b) career decision making; and (c) academic 
achievement. Since the studies reviewed by Cnderlein in the domain of the effect of 
career education on increasing aradcmic achievement are included in the set of studies 
found in the hhaorman monograph summarized earlier, they are ehininated from this 
summary. 
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Thrw impressive uwnmane^ of ddta tound in the Enderlei.i monograph illustrate 
the problem of "fugitive literature" facing those who atte;npt to provide a compre- 
hensive sumn.ary of the literature related to evaluation of career education. Those 
three reports are referenced m the Enderlein monograph as follows: 

Tuckman B.W. & J.A. Carducn Evaluating Career Education; A R eview 

And A Model. Report No. J6. National Institute of EducatlonTDHFW). 
Washington, n.C: U.S. Governinef»t Printing Of f ice, 197i». 

nevelopment Associates, Inc. Evaluation of V oca tional Exemplary Proie rts- 
Part D, Vocational Education Amendments o7T968. Prepared for fhT 
Division of Vocational and Technical Education, U.S. Office of Educa- 
tion, U.S. Office of Education, March. 1975. 

Behavioral Research Associates Coc hise County Career Education Prn|.rr, 
Evaluation Report. 197»-7 5. Tucson, Arizona, 1975. ' 

None of these three reports ,s readily available today to those seeking original data. 

Similarly, the Enderlein monograph itself may well also become largely unavailable 

within a few years. For these reasons, it seems appropriate here to reproduce the 

basK. tables found in the Enderlein monograph from each of these three reports. 

They appear here exactly as in the Enderlein monograph. 
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Author and 
Site of Study 

Cocfuan and Wci« 
(1972) Dayton. Ohio 



HoUlcin(1972) 

Lincoln Co., Wcit Virginia 

Ovard(1973) 
Utah 

Sims (1973) 
CIcvcbnd, Ohio 



McNuIty (1974) 
MaiMchuscttj 

Wirrvn(l974) 
Kan^ 

Barlolo{1972) 
Alameda Co., Calif. 



Grade 
Level 

9-10 



1-6 
6 

5- 6 

IM2 

6- 8 



TABLE 1 

TUCKMAN AND CARDUCCI Rt LEVANT 
TO CAP£ER AWARENESS AND S>ELF AWARE^^^^ 



Elcm, 



Instrument 

Ohio Vocat ional 
Inicrost SMrvcy 



Analysis 

Analysis of 
CO variance 



rjomcmadctestof Analy.^i.sof 
Oceupalional awareness cova/iancc 



Not specified 

Job information 
questionnaire 

Inventory 

Cgrce t MiUnrU j^ 
inventory 

Occupotion al 
jnf'ormation Survey 



Chi square 
Analysis of variance 



Anulysis of 
covy/iiuicc 

t-lcsi 



t«test 



I'^indings 

. .xtudenis from the vocational school 
were better informed about career 
choice. , 

**- . .career education students outper- 
formed control students on aU mcasurei'' 

. .students showed favorable change ir. 
attitude toward work. . /* 

. .treatment schools students had 
acquired more job information than the 
control." 

. .positive cnrreljillon between scores 
lor uU students. . 

. .Niiifiifkant dilTcrcnccs in mcjn 
scores in favor of trealinenl group." 

. .yielded greater tains for treatment 
Icvcl^" controls at clomontory 



ro 
1 
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(Table 1 - continued) 



Cunningham (1973) 
New Britain, Conn. 



EJcm. 



How I Sec Myself 
Scales ^ 



Dennaxd (1973) 
CbytonCc.Ga. 



Student Knowledge 
ol C'areers 



HoUkn (1973) 
Sou in Caroliru 



Peck (1973) 
District of Columbia 



Elcm. 



K-9 



A battery of 
cognitive and affective 
mstrumcnts 

Self Observatio n 
Scale 
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Work Attitude 
Surv ey 



Young (1971) 
New Or lean I 

Harmond (197 3) 
Hairing ton, Maino 



7-9 



6-9 



1-8 



Occupational Valuer 
I nventory 

Career Aw a t c n c s 

bevoh>xinTerrt" 

inventory 

Attitude Toward 
Work Inv entory 

Occupational 
Know!ed^:c Test 



•\ , .showed significant differences 
b»,»wccn the control and experimental 
groups/' 

*\ . .fourth graders in program had 
greater knowledge of career than those 
not in proi^ram/' 

"Kershaw and Chesterfield counties were 
fairly successful in efforts to teach 
elementary sclnol students about careers^' 

. .showed CDFP children to have 
better self-concept than controls." 

. .revealed CDEP students had more 
positive attitudes toward work than 
controls.** 

**. . .sUidcnts were realistic in their 
career planning." 

**. . .students were able to relate school- 
learned skills to work MUtn!ions;it 
reasonable level of proficiency." 

**A si>»nificant positive gain was found 
for students in grades 1-8." 



**. . .signillcjinl in mean scores of 
seventh grade students," 



Development Associates, Inc. 



TAB LI- 2 



Summai/ of Cmidusio^^ for Outcome Qucstioni Across All Projects by Student Group* 




Arc itudcnl pankipdnti able to ulcntify a greater 
number of occupations than non*participanti? (Ql) 

Do nudcnliilemonstralc more famiiiaiity with 
tisks and functions associated with selected occu- 
pations than the compaijson group? (Q2) 

Arc student participants more familiai with the 
icquisil ioni associated with selected occupations 
than the compaiison gioup? (Q3) 

Do student participants score hiplier on pre- 
vocational. Job readiness tests than the comparison 
group? (Q4) 

I)o student participants indicate more positive 
altitudes toward employmciit than non- 
poiticipants? (05) 

Is iJic varivly of cwrccrs being considered by 
individu*d p;utiupaling sUidcn is greater than that 
of studcnti in the u>mp;iils(>n t>roup? (Q6) 

l>i) more student iiarlklinmtx indimle huvinK a 
carver plan ilian the coiiipuxison group? 

Do more student participjnts cite their career 
preference as Dieir expected career than non* 
participants? (QIO) 



Yes 



No 



Yes 



Yes 



Yes 



No 



9th 
Grade 



Yes 



Yc!> 



No 



No 



No 



No 



No 



12th Grade 



Participating 
Teacher 



Yes 



Yes 



No 



No 



Yes 



No 



Yes 



No 



Counseling 
Group 



Yes 



Yes 



No 



No 



No 



No 



No 



Yes 



Work 
Experience 



No 



No 



No 



No 



No 



No 



No 



No 



Skill 
Training 



No 

No 

No 

No 

No 

No 
No 

Yes 



qERiC''^"' '^'"'""^'"^ ^" f^'^' PJtlpipanls for any of the criteria used to assess the 
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TAULt 3 

COMPARISONS OF STUDENTS WITH HIGH AND I OW l-XP0S4.Rt 10 CARt tR 
EDUCATION ON SEVtRAL DIMENSIONS RELATED TO THE ARIZONA CAKLl-R 
EDUCATION MATRIX ^ 



High ['Nposure Low Fxposure 

Sample Sa mple i-te^t Sigruticance 



Secondaiy Sludcniik 



Educational Awareness 

Knowicdjic of Skill Req. 
Knowledge ol Factors Con- 

Uibuting to Job Sutisfac- 

tion 

Common Threads in Jobs 

Economic Awareness 
Avwarcness of Career Mobility 
AwareneM of Factors Influ- 
cncbig Occup. Structure 

Decision Making 
Employ ability SkiUs 
Appreciations and Attitudes 
Knowledge of Career Clusters 
Score 

Interest in Career Clusters 
Score 



Educational Av^atcness 

Knowledge of Skill Req, 
Knowledge of Factors Co rv 
tributing to Job Satisfac- 
tion 

Common 1 Krcads in Jobs 

Economic Awareness 
Decision Making 
Appreciation and Attitudes 
Knowledge of Career Clusters 
Score 



46.8 


44,3 


3.01 


0,001 


54.5 


49.6 


2.40 


0.01 


63,8 


55.7 


3.40 


0.001 


6S.2 


57.0 


4.64 


0.001 


53.1 


48. 5 


1.83 


0.04 


40.9 


34.0 


2.78 


0.003 


53.2 


47.7 


2.27 


0.002 


35.3 


30.2 


2.94 


0.001 


26,6 


24.4 


3.04 


0.001 


45.3 


37.0 


3.90 


O.DOl 


34.2 


30.8 


1.85 


0.04 


25.9 


24.7 


1.97 


0.025 


Uaiy/lniermedute Students 






50.1 


48.5 


1.86 


0.05 



59.4 



50.5 



3.45 



0.001 



54.6 


48.5 


2o7 


0.005 


65.7 


59.8 


2.12 


0.01 


58.2 


52,6 


2.24 


0.01 


61.5 


54.7 


2.67 


0.005 


60.2 


54.9 


2.01 


0.020 


49.9 


44.8 


2,28 


0.01 
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In addu.on to the dat. reproduced above, the Enderlem monograph provided 
br.el ,u.mar.e, of f.nd.ngs - almost aii of which were highiy positive - from severai 
other studies evaluating the effectiveness of career education. However. Enderlein's 
summary of f.nd.gs f.ied to prov.de any detailed information regarding the specific 
career educat.on "treatment that was .nvolved. the nature of the experimental de- 
Sign, or specie findings and/or conclusions. For these reasons, no attempt Is made 
here to .nclude yet another table summarizing such studies. The following quotes 
from the "Summary" found m the Ender.ein monograph provides a general 
llavor w.th respect to the studies included in his review: 

ga;ns^ro"cciat;ot";^f:rmf^^^^^^^ 

knowledge of th^orld of wo r^nH r . f T^'"^ '"^turity, 
Findings relat ng to the eflect^nr./"^''''/^"^^ planning. 

educat on projects at both th» .i-,v.-Z. wno naa participated in career 
able to make dec s.^^s wficR r^u^red .nL '^""^^'•y ^'^els were better 
interests, of occuSni roles^^nH t^^^ "l'^*"" ""^s and 

Overall. Ihe evaluS e forts to da^ °' '° P'^"'"" 

varied «oects of caJeer education ^ ' '''''''' '"^"^ ^"^^ 

certainly, the br.ef descriptions found in the Enderlein monograph appear to be sup- 
portive of the general conclusions quoted above. However, because the actual studies 
themselves were unavailable for rev.ew at the time this monograph was written, 
.t seems inappropriate to comment further on the content, of the Enderlein 
monograph. 
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Datta, Lois-ellm et dl ^r^er_ga^uor^ Have That It Works? 

Washington, D.C.: U.S. Government Print ini^fkirT977. 
At the Commissioner's National Conference on Career Education held in Houston. 
Texas .n November, 1976, a major theme session was conducted using the .itle of 
this monograph. The monograph itself co^uains the formal statements made by presenter, 
at this theme session. 

The monograph begins with an introductory paper written by Dr. Lois-ellin 
Datta of the National Institute of Education. This paper is devoted primarily to a 
series of short critiques of the remaining papers presented. The concluding portions 
of the paper concentrate attention on some of the major philosophical/policy impli- 
cations associated with efforts to evaluate career education. It is a very thoughtful 
paper and one that deserves much wider attention than i, has apparently received 
to date. 

The second paper, written by Dr. Elvis Arterbury, reported on two studies of 
the effectiveness of career education at the fifth grade level - one concentrating 
on the effectivene.ss of career education in increasing achiever^ent in Language Arts 
and Social Studies and the second asking this sa.ne basic question with respect to 
academic achievement in mathematics. The Arterbury paper produced generally 
favorable evidence for both of these studies indicating .hat a career education 
"treatment" can produce significant gains in basic academic achievement among 
elementary school pupils. The Datta critique of this paper noted that these studies 
failed to distinguish that part of the "treatment" that associated itself with mfusmg 
career education concepts from that part associated with the instructional content 
itself. This IS a ma,or problem associated with a career education "treatment" oriented 
primarily arou/.d an jnfusion emphasis. That problem will be examined in some detaii 
later m this mofiograph. 
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The paper written by Dr. Frank Rapley for this monograph described his attempts 
to evaluate a K-i2 career education effort in the Jefferson County* Kentucky School 
System. While, due to lack of complete data due to problems associated with imple- 
menting a desegregation order, he was unable to perform all the statistical analyses 
he had hpped for, the data presented in this paper show gains in both reading and 
mathematics achievement for pupils exposed to the career education treatment. 
The most fascinating portions of this report are seen by studying the approaches 
being tried to evaluate a total K-12 career education effort rather than simply portions 
of that effort. The Datta critique of this paper warned readers not ^o make strong 
inferences concerning the worth of career education due to the weaknesses in the 
experimental design made necessary by the school reorganization problem. It also 
commended this paper to those interested in a total K-12 approach to evaluating 
career education. 

One of the clearest - and best - examples of a report evaluating career education 
is seen m the paper prepared by Dr. Phil Spieth of the Dade County, Florida Public 
Schools. Dr. Spieth's description of the career education "treatment" is both clear 
and complete. While the particular 'treatment" being described will not be one that 
all school systems will want to endorse, at least the description of what was done 
in the name of" career education" is made clear. While differences between experi - 
mental and control puptis reported in this study were not large, they were uniformly 
m directions favormg those pupils who had been exposed to career education. The 
D«>tta critique of this paper applauded the use of trend data over a period of years 
which Dr» Spieth had used. The clearest findings in this study were those indicating 
that the most favorable results are associated with those car.eer education efforts 
th^t have been in operation for the longest period of time - thus emphasizing, once 
again, the importance of evaluating career education in a longitudinal design. 



Dr. Richard Ru(f, Arizona State Department of Education, presented a paper 
(or this series sumnnanzing major Statewide findings from Arizona K-12 school 
-iystems relative to evaluation of career education efforts during the 1971-73 
period in the State of Arizona. This paper indicated that students with "high ex- 
posure," as Opposed to "low exposure," to career education: (1) have a greater 
knowledge of the wide range of occupations available to them? (2) have an increased 
awareness and understanding of educational requirements for various careers; 
(1) have a better understanding of skills and abilities required for various careers; 
and W have an increased understanding of economic rewards associated with 
various careers. In critiquing this paper, Datta pointed to the need to assess in- 
dependently the inte nsity and the quality of career education efforts - i.e., to 
separate out evaluations of *'how much" from evaluations of "how well." 

F'lnally, Dr. Sidney High presented a paper summarizing results from a small 
number of evaluation studies, each of which produced positive results indicating 
the worth of career education. The prime "message" found in Dr. High's papei 
is that career education CAN wOrk - in spite of the fact that many things currently 
being done under the name of "career education" are so weak as to almost surely 
fail to work. Of these studies (1) three studies show evidence of improved academic 
skills; (2) five of improved work values; ()) four of improved career decisionmaking 
skills; {-^ two of improved occupational and interpersonal skills; (5) fourteen of 
greater knowledge of educational/vocational opportunities; and (6) one indicating 
that career education provided studentN with an increase in the sense of control 
over their Uves. Datta's critique of this paper reminded us that, positive as these 
data are, they will certainly not suffice to convince the true skeptic of the worth 
of career education - due, in part, to the fart ihat only short-term outcomes of 
career education wore measured. 
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Bonne t , Debor ah C . A Synthesis of Results and Pro^rdmmdtic Recornmenddtions 
Emerj^tnj^ From Career Education Evaluations in 1973>76. 
Washington, D.C.: U.S. Government Printing Of f ice, 1978. 

During 1976, 0£' s Of f f ice of Career Education awarded a grant (000760^*329) 
to New Educational Directions, Inc. whose purpose was to synthesize and summarize 
evaluation results available from the 108 1? career education demonstration 
projects funded by the U.S. Office of Education during the 1975-76 academic 
year. A total of of these program^ were in their third year of funding under 
provisions of Sec. U2(c), Part D of P.L. 90-576 while 61 were operating as one 
year projects funded under provisions of Sec. ^^06, Title IV, P.L. 93-380. Of these 
108 projects, a total of ^5 ^ including 26 Part D and 19 OCE (sec. '*06) - contained 
evaluation data judged suitable for use in this project. These U5 studies were 
selected using the following critiera: (a) the report provided sufficient information 
about the measurement instrument for determining which outcome the study 
addressed; (b) the rep» . t indicated which of four grade-level groupings the students 
represented; (c) the outcomes of the career education group were compared to 
some reference group; and (d) the report indicated the direction of differences 
between means. There was no requireitient that results be analyzed inferentially 
nor that the number of students, the sampling pian, or the specific student career 
e- ucation activities be identified. 

The unit of analysis used m reporting results was called the "outcome study." 
An '^outcome study" wa> arbitrarily defined as the assessment of one learner out- 
come objective at one grade level. For example, an evaluation investigating a 
K-12 program's impact on reading achievement, work attitudes, and career knowledge 
at each of Grades ^. 8. and 12 would, using this definition, consist of a total of 
9 "outcome studies." lising this defmition, a total of ^00 ► "outcome studies" were 
classified under one of the 10 "learner outcomes" found m the OCE monograph 

P 



ERIC 



-21- 



entitled Per«)e<.tive On The P roblem of Evaluation In Career Edut^.^^tinn. Thus, 
a separate section ol this report appears for each of these 10 "learner c . comes." 

Treatment; Bonnet was unable, with the data available to her, to specify with ' 
any kind of exactness the particular kind<s) of "career education treatment" that produced 
the results reported in this monograph. As common traits, she listed the facts 
that all programs: (a) involved the development of materials, training of school 
staff, and involvement of the community; (b) shared the goal of preparing individuals 
for participation in the world of work; and (c) appeared to have embraced the 
basic definition, philosophy, and concepts of career education expressed in the 
1975 USOE pr.licy paper. An Introduction to Career Education. Within these broad 
common elements, she discovered very wide variation to exist in such important 
treatment elements as.- (the proportion of students who had received whatever 
career education "treatment" was being offered; (b) the length of time the "treat- 
ment" had been applied: (c) the definition of "infusion" used in specifying the "treat- 
ment"; and (d) the extent to which elements in addition to "infusion" within regular 
classrooms were included in the definition of the "career education treatment." 

Thus, while the results reported in this monograph can be said to reflect a 
career education "treatment" with some common philosophical meaning, they 
cannot be said to reflect a career education "treatment" with some common 
operational activities meaning. Apparently, the best that can be said is that some 
bonofide career education implementation strategies were used, but it is not 
known which strategies produced any given reported result at any specified levej 
or length of application. The results reported in this monograph must be viewed 
given this severe restriction of meaning with respect to the exact "career 
tion treatment" that was used. 

Design: The intent of this project was to synthesize evaluation results related 
to each of the 10 OCE "learner outcomes" as found in the 500* "studies" identified 
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by the author. A necessary first step was to make subjective judginents regarding 
the extent to which the various measurement instruments used related to one 
ot these 10 ''learner outcomes." That is, no attempt was made to infer that t^e 
measurement instruments were designed to be direct measures of each "iearner 
outcome." For some, the relationship is quite obviOus while, for others, it is not. 
For example, there were fairly direct measures used to assess "knowledge of oc- 
cupations" but, m heu of direct measures of "self-understanding," measures were 
used intended to nieasure "self esteem." 

:^or each of the 10 "learner outcomes," the following data were gathered and 
reported; (a) number of projects assessing that outcome; (b) th«^ -^oproximate 
numbers of students involved m assessments; (c) the number of "studies;" (d) the 
grade level categories (KO, ^^-6, 7-9. 10-12) used; (d) the type of research design 
used (e.g., "pre-post without comparison," "post with comparison"); (e) the number 
of "studies" with positive results significant at the .05 level; (f) the number of 
studies With positive differences between means; and (g) the probability that positive 
and negative differences are equally likely. 

Further, the design called for a table specifying each of the measuring mstru- 
ments 'ised, the number of times the instrument was used at each of the grade 
levels, the number of times statistically significant results were obtained, and 
the number of times it produced positive differences between names. Those data 
are reported m an Appendix of this monograph. 

The 10 OE "learner outcomes" were, in some instances, sub-divided into various 
"sub-outcomes'* where, in the opinion of the author, this was justified. The following 
represent the "learner outcomes" used as major categories for summarumg results 
of the 500 ♦ "studies:" 




Learner Outcome //I -a; Competent m the basic acddemic skills required for 
ddaptabiUly in our rapidiy ctianging society (READING ACHIEVEMEINT) 

Learner O utcome 0i-bi Competent in the bdSic academic skills required for 

ddaptdbilily in our rapidly changing society (MATHEMATICAL ACHIEVEMENT) 

Learner Outcome //2; Equipped with good work habits (ATTITUDES TOWARD 
SCHObD 

Learner Outcome &h Equipped with a personally meaningful set of work values 
thai foster in them a desire to work. 

Learner Outcome //»-a; Equipped with CAREER DECISION-MAKING SKILLS 

Learner Outcome //»-b; Equipped with JOfVHUNTING and JOB GETTING 
SKILLS 

Learner Outcome //Sa; Equipped with JOB SPECIFIC OCCUPATIONAL SKILLS 
and interpersonal skill? at a level that will allow them to gain entry into 
aid oOtdin a degree of success in the occupational society 

Le*, n e r Outcome //5-b; Equipped with job specific occupational skills and 
INTERPERSONAL SKILLS at a level that will allow them to gain entry- 
into and attain a degree of success in the occupational society 

Learner Outcome //6-2; Equipped wjth a degree of SELF-UNDERSTANDING 
and understanding of educational-vocational opportunities sufficient for 
making sound career decisions 

Learner Outcome //6-b; Equipped with a degree of self-understanding and 
UNDERSTANDING OF EDUCATIONAL-V<X:aTIONAL OPPORTUNITIES 
sufficient for making sound career decisions 

Learner Outcome tf7; Aware of means available to them for continuing and 
recurrent education 

Learner Outcome //8; Either placed or actively seeking placement in a paid 
occupation, in further education, or m a vocation consistent with their 
current career decisions 

Learner Outcome tf9: Actively seekmg to find meaning and meaningfulness 
through work in productive use of leisure time. 

Learner Outcome IQ; Aware of means available to thernselves for changing 
career options - of societal and personal constraints impinging on career 
dlterndtives 

Results: A "bird's eye view" of the extensive results found m this monograph 
are summarised, with respect to each of the "Learner outcomes," m the table ^' 
which appears below. 
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Table ; Summary of Results Obtained from 500* "Studies" 
Reported in the Bonnet Monograph 

Learner % "studies" with % "studies" with Probability that positive 
Outcome ' Positive results positive differences and negative differences 
at >_Q5 level between means are equally likely 

1-a 18% <>i% NS 

1-b 32% 52% NS 

2 10% 0<f% NS 

5 68% Less than .02 (♦) 

29% 70% Less than .001 (♦) 

<*-b Only 6 studies reported - one with statistically significant findings 

^a ^ot measured by career education. Perceived as a vocational education 

problem. 

^ 3% 61% Less than .05(0 

6-a(l) 9% 37% NS 

6-b(2) 15% 38% NS 

7 Only I study reported - it was positive 

« TOO EARLY TO EXPECT CONCLUSIVE DATA REGARDING THIS OUTCOME 

9 Only I study reported - it was positive 

10 Only 4 studies reported - 3 of which were positive 



In addition to the kinds of summary statistics illustrated in the table above, 

results are also summarized, in more detail<?d form, for evaluation studies associated 

with each learner outcome goal that produced positive results. An example of 

this kind of reporting can be seen in the following quotation referring to one of 

the studies evaluating the effectiveness of a career education infusion approach 

in the classroom on improvement of basic academic skills; 

"Perhaps the most encouraging results in the basic skills area were shown by 
a Part D urogram in Opelika, Alabama (2), where the infusion of career educa- 
tion into the curriculum involved ALL teachers and affected ALL students 
in the school system. In the spring of 1973 and again in the spring of 1976, 
all of the approximately 350 students in each of grades 3, 6, 8, and 9 were 
administered the reading and mathematics subtests of the appropriate levels 
of the California Achievement Test. In reading, system-wide average scores 
dropped a small but significant amount at grades six and nine but improved 
significantly at grades three and eight. The improvement at third grade 
was quite dramatic, with the 1976 third-graders' mean raw score (60.3) thirteen 
percent higher than that of 1973 third-graders (53.3). Mean scores in math 
improved at all four grade levels. Third* sixth, and eighth grade improvements 
were statistically significant and impressively large— thirteen percent at third 
grade, twelve percent at sixth grade, and twenty-two percent at eight grade." 
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A total ol [f^ career education demonstration projects are referenced specifically 

one or more times with descnptionii of their results usmg a format similar to 

that given in the example above. 

Conclusions: Bonnet includes a U2 page discussion/conclusions section 

for each of the 10 learner outcomes. Near the end of the monograph, she aUo 

provides a i-2 sentence conclusion statement with respect to each learner outcome. 

These 1-2 sentence "conclusions,'* as given by Bonnet, are reproduced below: 

Career education seeks to produce individuals,who, when they LEAVE school 
(at any a^e or at any level) are; ~ 

K Competent in the basic academic skills required for adaptability in our 
rapidly changing society 

CONCLUSION: Overall, results are neither positive nor negative but in 
selected instances improvements were both statistically 
and educationally significant 

2, Equipped with good work habits 

CONCLUSION: Measured attitudes toward school were affected only in 
isolated mstances. Data concerning work habits are 
largely lacking 

3. Equipped with a personally meaningful set of work values that foster in 
them a desire to work 

CONCLUSION: Good evidence of success has been demonstrated in 
strengthening youngsters* desire to work, 

^, Equipped with career decision-making skills, job-hunting skills, and job- 
gottmg skills 

CONCLUSION: Career decisiQn-making skills were strengthened with 

great consistency. Data concerning job-seking skills are 
sparse but somewhat promising 

^. Equipped with )ob specific occupational skills and interpersonal skills at 
a level that will allow them to gain entry into and attain a degree of suc- 
cess jn the occupational society 

C0Ni.:LnsiON; For several reasons career education evaluations rarely 
evaluate job specific occupational skills. Results 
of interpersonal skills evaluation are encouraging but 
are not conclusive 

6. Equipped with a degree of self-understanding and understanding of educational- 
vocational opportunities sufficient for making sound career decisions 
CONCLUSION: Self-esteem is rarely affected as it is currently measured, 

and no objective data of self-understanding were reportea. 

Career awareness was achieved in the overwhelming majority 

of cases and there is reason to believe that the intent 

of the goal is being met on a broad scale 
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7. Aware of means available to them (or continuing and recurrent education 
CONCLUSION: Evaluation results are lacking 

8. Either placed or actively seeking placement in a paid occupation, in further 
education, or In a vocation consistent with their current career decisions 
CONCLUSION; It is too early to measure career education's influence on 

this outcome on a wide-scale basis, 

9. Actively seekmg to find meaning and meaningfulness through work in productive 
use of leisure time 

CONCLUSION: Only one study was reported; its results were promising 

10. Aware of means available to themselves for changing career options - of societal 
and personal constraints impinging on career alternatives 

CONCLUSION: Evaluation results are sparse but very encouraging 

Discussion; The Bonnet monograph represents a valuable addition to the literature 
concerned with evaluation of career education. It appears that she did a sound 
and credible job with the data available to her. Results in this monograph are 
especially pertinent to those making funding decisions with reference to career 
education at the federal level because each project is one funded with federal 
career education demonstration funds during the 1975-76 academic year. It is 
partly for this reason that a thoughtful discussion of t \e contents of this monograph 
are in order. 

First, it is important to note that the OE "learner outcomes" for career education 
were designed m order that they might be applied to individuals at the point when 
such individuals leave the formal education system after having been exposed 
to a comprehensive K-12 career education effort. Each is obviously not intended 
to be applicable to students at any particular grade level within the K-12 education 
system. Thus, it is not surprising to find that, for the many "studies" Bonnet reported 
in which the participants were elementary school students, no clear results were 
in evidence. ' 

Second, it must be noted that Bonnet emphasized the measuring instruments 
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*v.iUbie to her for use were, by and large, not comtructed so « to be direct measures 
of these 10 "learner outcomes." Instead of having such measures available, she was 
forced ,0 use measures which, m her opinion, had some indirect relationship to a 
given "learner outcome.'* 

Third, .t IS essential to point out that, m actuality. Bonnet was unable, because 
of the nature of the data made available to her, to specify, with any degree of exact- 
ness, the nature of the career education "treatment" being evaluated. Neither was 
She able to determine how long whatever "treatment" existed had been applied nor. 
.n many instances, whether or not it had been fully apphed to ALL students included 
m the studies reported. It must be remembered that, during the 1975-76 academic 
year when these data were being collected, there had yet been no agreement with 
respect to what the "career education treatment" actually includes. There are avail- 
able in her data assurances that jome kind of career education "treatment" took 
place - thus justifying the anaiysis which she did. However, it would be extremely 
dangerous to take Bonnet's findings and conclusions as representing solid generali- 
zations with respect to the worth of these K.12 career education efforts. It was 
simply the best she could do with the data made available to her. Bonnet acknow- 
ledges this m the body of the monograph. 

Fourth. Bonnet's analysis illustrates very well (a) the need 5or measuring in- 
struments more appropriate for use in evaluating career education's effectiveness, 
(b) the need to insist on sound evaluations for all Federally funded career education 
demonstrafon projects (note that only *5 of the 108 projects involved met the minimum 
criteria Bonnet established for inclusion m her analysis), (c) the need for evaluation 
systems for use m career education that do not depend on criteria related to outcomes 
expected only after a complete K-12 effort, and (d) the need for evaluations to be 
based on the presence of a well defined career education "treatment" that extends 
beywid one year in length. Each of these problems represents a serious current 
challenge for career education. 
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In spite ol all the limitations obviously assv^'riated with both the data and 
the basic methodojogy Bonnet employed, her findings, overall, paint a generally 
positive picture with respect to the effectiveness of career education. Certainly, 
the evidence concerning career education's ability to positively affect (a) increases 
in students' desire to work, (b) increases in students' acquisition of career decision- 
making skills, and (c) students' awareness of the nature of occupations and the 
occupational society appear, even under these severe limitations imposed by the 
data available to Bonnet, to have been adequately demonstrated. Further, other 
than in the case of the one learner outcome judged inappropriate for use in evaluat- 
ing career education and the one outcome regarded as inappropriate for use at 
this stage in the history of career education, all other "learner outcomes" produced 
some statistically significant positive results. In short, while these data do not, 
in any instance, support a claim that career education will produce such positive 
effects, they do support a claim that career education can produce positive effects 
given the right circumstancc.^ The trends are, by and large, positive. At this 
early stage in the career education movement, this is most encouraging indeed. 

The one ^'learner outcome" for which, to me, the data were most discouraging was 
the one related to career education's effectiveness in increasing students' achievement 
in the basic academic skills. Even here, the data support a contention that a career 
education "treatment" can produce positive results. The problem, I suspect, is two-fold 
here. In part. I strongly suspect that, in many instances, the full career education 
"treatment" (as outlined in the OCE monograph entitled A Primer For Career Educatio n) 
was probably not applied. In part, I suspect it is a case of not applying that treatment 
to all pupils for a long enough period of time. Thus, while 1 am disappointed that these 
data did not appear more positive, I am not discouraged. I continue to remain firmly 
convinced that we will, given the proper application of the career education "treatment" 
be able to demonstrate career education's effectiveness in improving basic academic 
skills - especially where elementary/middle school pupils are involved. 
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HamiJton, Jack A. and Anit* M. Mitchell Pinal Technical Repofti Identification of 

E vduateo Exemplary Activities in Career 
Education (K- 12;. Paio Alto, r.ilf^ml.. 
American Institutes For Research, 1978. 

This document summarizes activities performed under a contract given the 
American Institutes For Research (AIR) bv OE's Office of Planning, Budgeting, and 
Evaluation. It reprtitnts the most comprehensive effort yet undertaken to identify 
and analyze current attempts to evaluate career education efforts by local K-12 school 
systems. The project called for AIR to identify those career education efforts that 
had been evaluated in a sound fashion, to identify the best of these, and to submit these 
to the Joint Dissemination and Review Panel (3DRP) for review and approval. The 
numbers involved in this AIR effort provide a good basis for judging the current 
"state of the art" with reference to the extent to which sound evaluations exist 
for existing K-12 career education efforts^ 

The project began with an attempt to Identify existing K-12 career education 
efforts with sound evaluation data. A nationwide search turned up nominations 
of 390 existing K-i2 career education sites. Of these, 257 submitted data to AIR 
for review. (It is, of course, not known whether the remainder did not have what 
they considered to be adequate data or whether they ignored the AIR request 
for materials). 

Three levels of review were used for judging materials submitted. At Level 
1, materials were evaluated on 7 preliminary criteria including, for example "the 
evaluation desij^n includes a '.omparijon group/standard of some kind," and "the 
evaluation report concludes that there is evidence of effectiveness," When submitted 
to this level of review. 64 of the 257 passed. Level 11 review was much more stringent 
especially in demanding that tests of statistical significance had been conducted 
for student outcome data. Twenty of the 60 sites passed Level II review. In Level 
III, a very stringent and comprehensive set of criteria were applied related to evidence 
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of elftctlv^ness. Only 1 1 of the 20 »:te» who had passed Level H review dhu passed 
Level III, Of those who passed Level 111 review, 10 were submitted to 3DRP and 
7 of those 10 passed* 

A brief ctptule summary of each of these 7 projects is presented below in outline 
formi 

1. Akron Career Development Program . Akron Public Schools - Akron, Ohio 

(a) Population! Grades 10 (60% jpinorlty in urban school district) 

(b) Treatment : Infusion in classrooms with strong community involvement 

(c) Deslem »0 experimental end <*0 control classes - end of year testing 

(d) Rfffumi Experimental (career education) students outscored control 
students in 35 of 36 comparisons of subgroups at the 3rd, 6th, 8th, 
and 10th grade level in the following cognitive areas: 

(1) knowledge of decision making 

(2) knowledge of "Individual and Environment" 

(3) Knowledge of self 

(4) Knowledge of employability and work adjustment skills 
(3) Knowledge pf the world of work 

(6) Knowledge of economic understanding 

(7) Knowledge of education a'ld training opportunities 

2. Developmental Career Guidance Project - Pima County, Arizona 

(a) Population t Grades <*-12 pupils in mixed rural/urban setting; 20% minority 

(b) Treatment ! Infusion of efforts into regular classrooms through leadership 
of counselors trailed to be career development specialists 

(c) Desiitm End of year cognitive measures to students in 26 schools with 
"high exposure" to career education and 25 schools with "low exposure" 

(d) Results: Statistically significant differences favoring experimental 
(high career education exposure) over control at each grade level in: 

(1) Career awareness 

(2) Economic awareness 

(3) Career decision making skills 
W Employability skills 

(5) Appreciations and attitudes toward work 

(6) Knowledge of career clusters for broad groupings of occupations 

3. Project CAP - Boston Mountains Educational Cooperative - Greenland, Arkansas 

(a) Population : Grades K-S students in poor rural areas 

(b) Treatment ; Infusion of career education concepts into regular subject 
matter by regular elementary school teachers 

(c) Design: Experimental/control classes of students, K-8, with one school 
designated as "experimental"' and the other as "control" 

(d) Results : Experimental (career education) pupils showed statistically 
significant gains over control students in: 
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(b) 

10 



ii) Scores on a Career Awareness Test 

(2) Scores on a Career Knowledge Test 

(3) Scores on an Orientation to Career Concepts/Worker 
Activities Scale 

Mi?lSgLi?^^ " ^""^^^ Community School District - Coloma, 

(a) Population: K-6 pupils in both rural and urban areas 
Freatment: 27 career education curriculum units to be 
infused into regular subjects 
Desi^: Pre/posl measures onl38 career education performance 
objectives 

(d) Results : Where sustained exposure oCcured, 95 of 138 performance 
Objectives were reached. Academic achievement in 
basic skills was not reduced and, in some cases, increased 

Project CERES - Ceres Unified School District - Ceres, California 

(a) Population; K-.6 pupils in rural community 

(b) Treatment; Infusion into regular ciassroomsj strong community 

involvement ' 
(cj Desifa;ni Experimental/control classes with pre/post tests 
fA\ o . and knowledge of careers 

W Result s. ! Statistically significant differences favoring experi^ 
mental (career education) pupils on cognitive tests 
of ''career awareness" and "knowledge of careers." 
Gains in academic achievement for low income 
pupils, but not for middle income pupils in experimental, 
as opposed to control groups* 

Project EquaJity - Highline Public Schools . Seattle, Washington 

(a) Population; Grades K^6 pupils - predominantly white middle^ 

class 

(b) Treatment; Development and infusion of 3 sets of materials 

into classrooms designed to reduce effects 
of occupational sex stereotyping in both girls 
and boys 

(c) Design; Experimental/control classes with pre/post testinc 

(d) Results; Experimental (career education) pupils showed 

statistically significant gains over controls in 15 
of 17 comparisons. 

(e) Conclusion; Given this treatment, pupils will expand their 

perceptions of job options open to females 
and to males 

Project MATCH - Ontario^Montclair School District - Ontario. 
California 

(a) Population ; K-8 pupils in urban settings 

(b) Treatment; Infusion of 1 10 career education objectives 

into regular subjects 
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(c) Desi&na Pre/post measures with experimental and control 

classes randomly assigned at each grade level 

(d) Results ; Statistically significant differences in cognitive 

Items of career development at each grade level 
favoring experimental (career education) pupils 
over control pupils. 

These seven "super" evaluation studies joined, m late 1978, with several 
earlier studies that had previously been approved by the JORP and submitted 
for inclusion in the National Diffusion Network (NDN) publication of EDUCATIONAL 
PRACTICES THAT WORK. They make a most significant set of additions to the 
literature related to evaluation of career education. 

One of the most valuable portions of this document consists of the list of 
ail projects nominated to AIR for consideration in this project. That list provides 
a clear indication that evaluation of career education is a matter of high concern 
on the part of literally hundreds of K-12 school systems involved in implementing 
career education efforts* The fact that most of these have not yet developed 
the kinds of sophisticated evaluation designs that enable them to pass the rigid 
requirements of the JDRP appears to be more a function of the early stages of 
evolution in career education than it docs a lack of interest in evaluation. 

In lookmg at these seven "super" evaluation efforts, it seems worth noting 
that: (a) a majority of them concentrated at the elementary school, rather than 
the secondary school level; (o) most of these positive results were obtained using 
an "infusion" rather a "special course" approach to implementation of career 
education; and (c) most of these studies used cognitive measures reflecting short- 
term, intermediate goals of career education. These observations hold important 
implications to be discussed in a different section of this monograph. 
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Summary oi thg Sijmmari>.« 

. Of the 38 stud.« renewed .n the Bhaerrnan monograph, 1 9 reported statistically 
Significant results favor.ng career education while an additional 16 indicated some 
support lor contending that career education is effective. When the three tables reprcv 
duced from the Enderlein monograph are examined, one can see positive result, reported 
for all 16 studies included in the Tuckman and Carducci report, for 6 of the 1 3 analyses 
made for 6th and 9th grade pupUs m the Development Associates report, and for all 
20 comparisons shown in the Cbchise County Arizona career education project. Other 
studies reviewed m the Enderlein monograph but not included here were also generally 
positive. 

If we eliminate from the "Career Education: What Proof Do We Have That It 
Worksr- monograph those studies included in other documents summarized in this mono- 
graph. I count a total of 15 additional studies, each of which reported some favorable 
conclusions regarding the effectiveness of career education. 

The Bonnet monograph, because of the unique way in which Bonnet defined 
an "outcome study,- car^not be directly compared with the other monographs summarized 
here. The total of 4 5 evaluation studies included in the Bonnet analysis produced a 
grand total of 552 -outcomes." of which 99 were statistically significant at the .05 
level of confidence and 352 showed positive differences between means favoring those 
pupils who had been exposed to career education. Of the U studies specifically referenced 
in the Bonnet monograph, all showed positive results demonstrating the worth of career 
education. 

Finally, the Hamilton and Mitchell document shows clear and conclusive evidence 
concerning the worth of career education for the seven evaluation studies that success- 
fully passed JDRP review. That same document clearly illustrates the need for better 
evaluation data in many other projects. 

Taken as a whole. ,t certainly seems safe to claim that the evidence summarized • 
.n this set of monographs points to a conclusion that K- 12 career education efforts 
can - and often do - produce favorable results. The overall o.cture is much more positive 
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th4n it IS negative. There are some reasonably good answers that can be given to those 
who inquire with respect to whether or not career education is effective. This body 
of evidence should be sufficient to answer initial questions raised by the true skeptic. 
It will not, however, adequately answer questions raised by the true "disbelievers." 
For such persons^ much more remains to be done. 

The persistent difficulties one observes in studying the documents reviewed here 
can be summarised easily. First, there are serious questions, for many of the specific 
studies, regarding exactly what has been tested in the name of "career education." 
Second, there are serious questions regarding the relatively short time frames in which 
many of the studies have been conducted. Third, there are questions regarding the 
appropriateness of the measuring instruments used for assessing several of the "learner 
outcomes." Fourth, there are questions regarding appropriate models for use in evaluating 
career education. Finally, serious questions remain regarding the exact kinds of "treat* 
ment" given the "control groups" in many of the studies reported in this set of docu- 
ments. 

In hope of providing those charged with continuing and expanding efforts to evaluate 
career education efforts during the decade of the l9S0s, let us now turn our attention 
to what appear to be the rnost basic problems facing us in evaluating career education 
efforts. 



issue l! The Career Education "Treatment" 

The single most important issue to be resolved in designing and carrying out 
career education evaluation studies relate to the need to clarify and specify exactly 
what has been done in the name of "career education." A host of major problems 
must be faced if this issue is to be resolved. 

First, m the basic OCE policy monograph, A Primer For Career Education , the 
"career education treatment'* is defined in terms of a total of 53 roles and functions 
to be earned out by eig.'it differcfit kinds of "actors" in the delivery of career educa- 
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tion. In order to ...e«t rt-spons.buu.^ for Federal leadership while simultaneously 
avoiding any implications of pos.«ble Federal control, these 53 roles and functions 
were written m term, of what is to be done in career education without specifying 
how each role and function .s to be carried out. In order for those who evaluate career 
education to specify, with a needed sense of exactness, exactly what is being done 
m the name of "career education." .t will be necessary for them to specify the "how" 
as well as the -what" aspects of the career education treatment. In brief, simply to 
assert that the career education treatment, as specified in A Primer For r.r..r P.,...- 
tion^ was carried out will not be sufficient. 

Second. It IS both unreasonable and undesirable to attempt to demand that even 
the 53 general role and function statements found in the OCE definition of the "career 
education treatment" be carried out before one could say that "career education" is 
being practiced. It is unreasonable in view of the fact that, as with any aspect of 
Education, a number of schools of thought exist with respect to defining the nature 
and goals of "career education." Like the term "evaluation" itself, different experts 
assign differing meanings to the words. It is undesirable in that, if everyone approached 
the delivery of career education in the same basic way, opportunities for discovering 
new and better delivery approaches would be lost. Thus, the 53 roles and functions 
proposed by OCE must continue to be regarded as only one of several possible ways 
of defining the "career education treatment." ^valuators are, however, under a clear obli- 
gation to specify exactly what they mean by the term "the career education treat- 
ment." Too many.to date, have failed to accept th.s obligation. 

Third. .1 the rationale behind the 5) roles and functions to be performed by the 
eight sets of "actors" pictured m APnmej^F or Career Education .s accepted. ,t ,s 
obvious that what ,s to be evaluated will be a community effort, not simply an effort 
of the Education system alone. There will surely be a host of problems associated 
with verifying tn. de^roe to which such a total community effort is present, let alone 
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evaludting its eflectivenc-.*. The data collection problems inherent in verifying the 
presefKe of the effort cledrly point to a concern relative to the evaluability of career 
education as a total community effort. Further, those who evaluate career education 
as a community collaborative effort will obviously have to do so based on concepts 
ol community accountability for the success - or lack of success - of career educa- 
tion. It seems obvious that, to reach the stage in any community where this require- 
ment can be met, a given career education effort will have to be in place for a period 
of several years. It would be completely unrealistic to attempt to evaluate career 
education m terms of these 55 roles and functions at the end, say, of only one year 
of operation- Still, some kind of evaluation of efforts at the end of even the first 
year will be called for. The model proposed in A Primer For Career Education will 
not be one that can be validly applied at the end of the first year of operation- 
Fourth, if these 53 roles and functions are studied carefully, it will be obvious 
that each of them is, itself, subject to evaluation. Further, it will be obvious that 
the proper ways in which most, if not all, of these 53 roles and functions should be 
evaluated will require evaluative criteria quite different m nature from the OCE 
"learner Outcomes" that are proposed for use in evaluating the total career education 
effort. At this point in time, proper evaluation designs for use in evaluating many 
of these 53 roles and functions have not been devised. Until they are devised and 
put mto operation, it will be operationally impossible to evaluate the total "career 
education treatment" in terms of the OCE "learner outcomes." 

Issue 2; The Need For Intermediate Criteria 

A second basic issue facing those concerned with evaluation of career educa- 
tion IS evident m the developmental, longitudinal nature of the career education 
concept Itself. The career education "treatment" is one that is expected to begin 
no later than when the individual enters the formal Education system (and, hopefully, 
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even before that point m time in terms of home/family actions). It ts expected to 
continue throughout the period of time the individual remains ir, the formal Education 
system. Further, it is expected to be a "treatment" that becomes available to persons 
at those times m their adult lives when they again have a need and/or desire to use 
Education as a means of helping to resolve work/education problems. In short, the 
career education "treatment" is seen, coiKeptually, as one that should be available 
to persons throughout almost all of the life cycle ^ including the retirement years. 

In order* to redute this broad, philosophical view to one that is more nearly sut>- 
ject to systematic evaluation, OE's Office of Career Education has described "learner 
outcomes" for career education in terms of competencies and actions the individual 
should be expected to demonstrate at that point in time whenshe/he leaves the formal 
education system for the first time. This time of leaving may, in the case of high 
school dropouts, be prior to completion of the secondary school. For many persons, 
that point in time will be represented by high school graduation. For many others, 
it Will be represented when one leaves a community college, a four year college/university 
system, or a graduate/professional school with an advanced degree. 

For purposes ot narrowing still further specification of career education "learner 
outcomes" to those that can be associated specifically with the K-12 portion of the 
Education system, the original "learner outcomes" specified in the OE policy paper 
An Introduction To Caree r Education has been further ref med and narrowed to the 
nine "learner outcomes" listed in OCE's latest basic career education policy paper. 
A Primer For Career Education . *hile. at the time that paper was written, these 
••learner outcomes" were still being defined as though they apply to persons leaving 
the Education system at any age or at any level, they have been put in a form that 
will allow their initial use to bo made at the point when the individual leaves the K- 
12 system of Education. The 9 "(earner outcomes" included in this more restrictive 
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view includei 

!• Competent in the b^ic academic skills required for adaptability in our 
rapidly changing society 

2. Equipped with good work habits 

3. Equipped with a personally meaningful set of work values that foster in 
them a desire to work 

^. Equipped with career decision-making skills, job-hunting skills, and iob- 
getting skills 

5. Equipped with a degree of self-understanding and understanding of 
educational- vocational opportunities sufficient for maicing sound 
career decisions 

6. Aware of means available to them for continuing and recurrent 
education 

7. Either placed or actively seeking placement in a paid occupation, in 
further education, or in a vocation consistent with their current 
career decisions 

8e Actively seeking to find meaning and meaningfulness through work in 
productive use of leisure time 

9. Aware of means available to themselves for changing career options - 
societal and personal constraints impinging on career alternatives 

The basic contention made in A Primer For Career Education is that, IF the 
53 roles and functions outlined for the "career education treatment" are carried out 
properly DURING THE ENTIRE K-12 PERIOD OF SCHOOLING, each of these nine 
basic "leainer outcomes" will be achieved. This is the basic contention that needs 
to be subjected to caref ul evaluation . To adequately test this contention will demand 
that data be collected for a given group of individuals, beginning at the Kindergarten 
level and continuing until graduation from high school, demonstrating that the com- 
plete "career education treatment" has, in fact, been adequately provided. Applica- 
tion of evaluation procedures for each of the nine "learner outcomes" should then be 
applied for persons who have received this verified "career education treatnent/' 

This kind of eva luation of the "career education treatment" has not yet been 
undertaken. Since the formal career education movement did not begin until around 
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l»70. it i. Obviously too soon to expect that thi, coyid have occured. TheoreticaJly. 
there should 6e at least a few K-12 school systems somewhere ,n the Nation who began 
their career education efforts in the 1970-72 period and have continued them ever 
«.nce. U such school systems could 6e located - and IF they had available the Kind, 
of data required for ver.fy.ng that the 53 roles and functions outlined In the "career 
education treatment'- have. ,n fact^een taKing place for pupils ever since they began 
« Kindergarten enroUes .n the 1970-72 period, it could be expected that the kinds 
of evaluation studies most needed could be conducted sometime during the 1982-8* 
P«r.od. Realistically, this simply cannot be expected to occur both because (a) the 
career education concept was not fully developed during the 1970-72 period, and (b) 
no current K-12 school system has the kinds of detailed data required to verify 
the complete "career education treatment- has taken place. ,t seems more reason- 
able to believe that th.s kind of evaluation study probably will not be possible until 
sometime during the decade of the 1990s. 

In the meantime, we will have to content ourselves with the use of only a portion 
of the nine "learner outcomes" and. .n many cases, with the use of intermediate criteria 
rather than the ultimate criteria represented by the "learner outcomes" themselves. 
Further, we will have to recognize that, in the case of certain of these "learner out- 
comes." it would be grossly inappropriate to apply them to pupils at the elementary 
school level. This, in no way. means that no sound evaluations of career education's 
effectiveness can be conducted at this point in time. Rather, it simply means that 
the career education movement .s still too young to be able to test its effectiveness 
•n attaining all nine of the "learner outcomes." Further, while we are not yet in a 
position to evaluate the effectiveness of career education over the entire K-12 period 
of Education, we are now in a position to begin the process of longitudinal evaluation 
studies extending over a per.od much longer than a single school yoar - or portion of 
a Single school y.ar. The decad the 1970s for career education was one characterized 
largely by short-term studies of the effectiveness of career education. The decade 
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ol the 1910s will, hopefully, be one where evaluation studies cover a period of several 
ye«ri« If this can be done, then the decade of the 199Qs can be one where evaluation 
studies cover the entire K-12 period of schooling. 

Issue 3t Finding and Uslnt Proper Control Groups 

It Is obvious that a variety of kinds of evaluation designs are needed in the evalua- 
tion of career education efforts. It is equally obvious that, among such designs, many 
persons will still place their highest values on the traditional "experimental/control" 
model which calls lor persons in the "experimental" groups to receive the ''career 
v^ducation treatment" while persorw in the "control groups" receive some other "treat- 
ment." Thus, if the traditional "experimental/control model" is not regarded as 
sufficient for demonstrating the effectiveness of career education, it will continue 
to be necessary ^ There are several problems involved here, none of which have been 
adequately recognized to date. 

First, it should be recognized that, in m^ny of the currently existing evaluation 
studies, there has been an almost complete absence of the description of the "treat- 
ment" gl\en to pupiln in "control" groups. There exists as great a need to describe 
the "control treatment" as to describe, in exact terms, the "career education treat- 
ment." It is hoped that future evaluation studies will recognize this basic fact. 

Second, it should be recognized that, in a broad, generic sense, the "career 
education treatment" demands that: (a) a sense of purposefulness be inserted into 
the teaching/learning process for both students and teachers - i.e., that students 
feel it is important to learn what is being taught and that teachers feel it is important 
to help students learn what is being taught; and (b) the broader community be used 
as a learning resource in the total educational effort. In the "career education treat- 
rrent,*' an emphasis on the importance of education as preparation for work - paid 
and unpaid - part of the person's total lifestyle is used as the basic vehicle for meet- 
ing those demands. The essential questions to be resolved here are: (a) should "control 
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iroupsi co.«ut of "treatment," where these two demands are not part of the "treat- 
ment'.? or (b) Should "control groups" consm of "treatments" where these two demands 
.re met through some vehicle other than the "careers emphas.s" of career education? 
M the fir\t of these two alternatives .s adopted, one would have to assume that the 
teaching/learning process holds httle meanmg for euher pupUs or the.r teachers and 
that the education system has isolated .tself from the broader community. While 
we know, of course, these conditions exiSt .n many K-12 school systems today, to use 
these as "control groups" will surely be perceived by many as "stacking the cards" 
in favor of shoving greater gains for "experimental groups" than for tontrol groups." 
IX may be the best thing to do. but .t .s almost certain to result in this kind of basic 
criticism, 

H. on the other hand, "control groups" are defined in terms of use of an alternative 
vehicle for, (a) making the teaching/learning process more meaningful to all teachers 
and ali pupils at all grade levels; and (b) involving the broader community in the total 
educational process, then the "career education treatment" would have to be very 
powe. /ul indeed if significant gains are to be shown for "experimental groups" as 
opposed to "control groups." Ideally, this would appear to be much preferable to the 
first alternative. The problem .s one of iden'ify.ng such an alternative vehicle that 
uses an emphasiS on something other than "careers" while employing either the same 
basic strategies or a set of different basic strategies than those used in career education. 

Those contemplating this alternative need to do so recognizing the basiC strategies 
involved .n the career education effort which have kept this effort going lor 10 years 
- three times as long as the typ.cal "educational reform" efforts that have come along 
in the past. Such strategies include: (a) promoting the ef foi t based on a basic need 
perceived as real, important, and continuing in nature - as seen by educators, students, 
parents, and the general public, (b) choosing a need that is applicable to ALL pupils 
at ALL grade levels and within ALL subject matter disciplines .n the education 
system, (c) choosing a need that influential community members Will acknowledge 
as real and recognize as one for which their expertise would be helpful in meeting; 
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(d) choosing d need that involves '*selijng" rather than "buying" teacheri - i»e.» a need 
which* if met, will enhance the instructional goals of the teacher rather than being 
nnet through adding new tasks and responsibilities that nnay interfere with attainnnent 
ol instructional goals; (e) choosmg a need that will not only gain parental support but 
will, in addition, give parents clear roles and responsibilities; (f) choosing a need which 
represents one of the basic goals of Education which, if nnet, will enhance attainnnent 
of other basic goals of Education as well; (g) choosing a need which can be met with- 
out greatly increased costs to the education systenn; (h) choosing a need which will 
result in an effort clearly susceptible to clear evaluations; and (i) choosing a need 
which holds potential for becoming institutionalized as part of the basic fabric of the 
education system itself. By concentrating on the need to help ALL pupils better 
understand and solve problems of work/education relationships, career education has 
been able to use all of these strategies. To date, no such alternative strategy that 
also meets all these needs appears to be present. 

As a result, the "career education treatment" appears to be one best compared 
with whatever else the school system was doing before the "career education treat- 
ment" started. If "experimental" and "control" groups are established in such a school 
system, it is obvious that it will be relatively easier to demonstrate career education 
to be 'Effective" in those school systems where low quality educational efforts existed 
than m school systems where high quality educational efforts were in place prior to 
the time a 'bareer education treatment" was instituted. In any event, the "control 
treatment" is fully as important to desc{ ibe and define as is the "career education 
treatment," 

Issue ^: The Teaching/Learninx Process 

The teaching/learning aspect of the "career education treatment" represents 
a sufficiently important part of the previous issue so as to deserve as additional 
separate discussion. 
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A Primer For Career Educatio n, a total of 10 roles and responsibilities are 
•Migned classroom teachers in the total "career education treatment." The follow- 
ing portion of these 10 roles and responsibilities are ' concern here: 

"I. :,eeking to improve academic achievement through using a "careers 
emphasis" as a vehicle to: * 

a. Introduce a sense of purposefulness and meaningfulness into the 
teaching/learning process for both student and teacher through 
emphasizing that one of the reasons for learning the subject 
matter is that people use it in their work. 

^' ^K*^^'" u""" ^PP''°^^^ ""'^^ students through rewarding students for 
what they have accomplished rather than emphasizing what they have 
failed to accomplish, how much more they have to accomplish, or how 
many other students accomplished more. 

c. Introduce variety into the teaching/learning process through utilizing 
the personnel and physical resources of the broader community as 
"fZ V 'niproving stuctent achievement - for emphasizing that 

n f !^\l^'"'i''^T ^^y'' ^'"^P'y "I'-o^gh r-ading. in more 

places than the classroom, and from more persons than certified class- 
room teachers. 

2. Consciously and conscientiously provide rewards to students who exhibit 

and practice such basic good work habits as: (a) coming to work (to 
school) on time; (b) completing assignments that are begun; (c) doing the 
best that they can; and (d) coooerating with one's fellow workers (students) 

3. Combining a cognitive and experiential approach in the teaching/learnine 
'Wmng to°do" ^""P^^^'-^'^S desirability of "doing to learn" and 

Helping students acquire decision-making skills through using a project 
act.vity-oriented approach, when appropriate, in the teaching/learninR 
process that allows students to actually enKage in the decision-making pro- 

5. Systematically attempting to reduce biases students may have with respect 
to race, sex. or handicapping conditions in ways that will maximize freedom 
oj choice for all persons. 

6. Helping students discover ways in which the subject matter being learned 
can be valuable to students in productive use of leisure time." 

The remaining four "roles and responsibilities" assigned classroom teachers in the "career 

education treatment' arc more specifu-ally oriented around the "careers" emphasis. 

The six listed above, *hile involving use of a "careers emphasis" as a vehicje, quite 

obviously pertain to a generic set of activities which good teachers have always 
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practiced. This is undoubtedly one of the reasons some oi career education's critics 
have complained that all career education advocates are asking for is '*good teach- 
ing." 

The question, in evaluating career education through the use o! "experimental" 
and "control" groups, is one of whether or not, for pupils in the "control treat- 
ment/' these SIX generic kinds of teacher activities are to be actively discouraged. 
If this were to happen, the net result might well be that the resulting evaluations 
would simply become a matter of comparing "good teaching" with "pcor teaching." 
This would not appear to be a fruitful course to follow - in spite of the fact that it 
would obviously make "career education" look good. The only viable solution that 
I can see is to depend on comparing the "career emphasis" as a vehicle for producing 
good teaching with whatever other vehicles are being used by "good teachers" in the 
"cwUrol groups." Once again, evaluators of career education are faced with a 
situation where it will be much more difficult to produce positive evidence of the 
effectiveness of career education m schools where "good teachers" already exist 
than in schools where they don*t. This is simply something that must be recognized 
and faced. 

This problem points up clearly the importance of conducting career education 
evaluations over a period of years with longitudinal data rather than in a period of 
a single school year - or portion of a school year. Any procedure designed to improve 
academic achievement is likely to be faced with this same problem - i.e., real gains 
in academic achievement are not likely to occur with a very short time period. 



The career education effort has been subjected to a wide variety of kinds of 
evaluation efforts during the decade of the 1970s - its first real decade of existence. 
On the whole, results of those evaluative efforts completed to date have been more 
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positive than n.g.tive. There .5 more reason io believe that career education is ef- 
fective than there .j to believe that it is lacking in effectiveness. 

At the same time, the decade of -evaluative efforts carried out during the 1970s 
h« made clear some of the major problems facing career education m demonstrating 
both its worth and its effectiveness. Some of those problems stem from the fact that 
the career education concept is still evolving - thus making any kind of final statement 
with respect to the "career education treatment" an unattainable goal. A part of the 
general problem facing career education is .ts longitudinal, developmental nature that 
is going to demand longitudinal studies before its true effectiveness can be assessed. 
Coupled with this is an obvious need to evaluate the presence of the "career education 
treatment" itself as well as the effects of that treatment. This means that a variety 
of evaluation designs must be put into operation. 

Finally, the greatest challenge facing those charged with evaluation of career 
education efforts will be that of spec.fy.ng exactly what is done in the name of the 
"career education treatment" and what is done with groups used for comparison pur- 
poses. This is a crucial problem to be solved if the goal of retaining local control over 
career education efforts is to be maintained. The goal of working toward local autonomy 
and away from any form of Federal control is worthy enough so as to justify whatever 
effort is required to solve tMs problem. 
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